Two Case Studies of Software Architecture for Multimodal Interactive Systems: VoicePaint and a Voice-enabled Graphical Notebook

This paper discusses software architectures of multimodal systems. The recent availability of new input technologies brought a whole new type of systems, able to support communication with the user through multiple interaction channels. Multimodal systems that allow modalities to be combined seem to be the most promising in the field of multimodal interaction. The aim is to overcome limitations of these modalities when used separately. We call synergic systems, systems which allow both different modalities to be used in parallel and modalities to be combined to obtain a command. VoicePaint and Notebook are two multimodal synergic systems developed by our team. We focus on their software architectures in this article.